Dynamical model of sequential spatial memory: winnerless competition of patterns 
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We introduce a new biologically-motivated model of sequential spatial memory which is based 
on the principle of winnerless competition (WLC). We implement this mechanism in a two-layer 
neural network structure and present the learning dynamics which leads to the formation of a WLC 
network. After learning, the system is capable of associative retrieval of pre-recorded sequences of 
spatial patterns. 



It is well accepted that the hippocampus plays the cen- 
tral role in acquisition and processing of information re- 
lated to the representation of physical space. The most 
spectacular manifestation of this role is the existence of 
so called "place cells" which repeatedly fire when an ani- 
mal is in a certain spatial location |[) . While much effort 
has been spent on experimental search and modeling of 
the so called "cognitive map" ||] as a paradigm for spa- 
tial memory, recent neurophysiological research favors an 
alternative concept of spatial memory based on a linked 
collection of stored episodes || . Each episode comprises 
a sequence of events which, besides spatial locations, may 
include other features of environment (orientation, odor, 
sound, etc.). Each distinct event is accompanied by time- 
locked activity of a certain hippocampal cell. Dynamical 
modeling of the emerging concept of the episodic mem- 
ory is of apparent general interest for neuroscience. Sev- 
eral models of associative sequential memory have been 
proposed in the literature Q. Most of them are based 
on the generalization of the Hopfield associative mem- 
ory network Q to include asymmetric synaptic connec- 
tions. Accordingly, they suffer from difficulties typical 
for Hopfield-type networks: the abundance of spurious 
attractors (sequences), complex structure of attractor 
basins, and sensitivity to noise. 

A dynamical model of the sequential spatial mem- 
ory should be based on the following experimental facts. 
First, there is a clear separation between neurons directly 
responding to specific stimuli (we call them sensory neu- 
rons, SN) and hyppocampal cells in CA1 and CA3 re- 
gions (principal neurons, PN). PNs fire in response to 
a combined vector of stimuli corresponding to a partic- 
ular event. Second, while sensory neurons are not di- 
rectly connected to each other, the PNs are coupled via 
inhibitory connections controlled by interneurons (INs). 
Third, the synaptic connections among PNs and between 
PNs and SNs exhibit Hebbian Long-Term Potentiation 
(LTP) H 0]. Based on these features of the hippocam- 
pal network, we propose a two-layer dynamical model of 
the sequential spatial memory (SSM) that can answer 
the following key questions, (i) How is a certain event 
(e.g. an image of environment) recorded in the structure 
of the synaptic connections between multiple SNs and a 
single PN during learning? (ii) What kind of the cooper- 



ative dynamics forces individual PCs to fire sequentially, 
which would correspond to a specific route (a sequence 
of scenes) in the environment? (iii) How complex should 
this network be to store a certain number of different 
episodes without mixing different events or storing spu- 
rious episodes? 

The key mechanism of storing of sequential memo- 
ries within our SSM model is the winnerless competition 
(WLC) Every event (image, odor, etc.) is repre- 
sented by a fixed point in the phase space of the model 
in which one PN is activated and others are silent. Ev- 
ery fixed point among those included in the sequence is a 
saddle with many stable separatrices and a single unsta- 
ble separatrix which connects it with a fixed point repre- 
senting the subsequent event. The resulting network of 
separatrices would form a complex heteroclinic connec- 
tion which would lead the system along the sequence of 
events in the specific episode. 

Let us discuss the learning objectives which would lead 
to formation of the sequential spatial memory (SSM). 
The first objective is to learn a projection map: as a re- 
sult of unsupervised learning the image of a particular 
environment (snapshot) encoded by heightened activity 
of the group of sensory neurons (SNs) leads to the height- 
ened activity (firing) of just one PN (see Fig.l). The sec- 
ond objective is to learn the temporal sequence of images. 
This can be achieved by modifying inhibitory connections 
among PNs due to long-term potentiation (see e.g. ||). 
The resulting structure of the phase space for the PN 
layer will exhibit features of the winnerless competition 
M. After the learning is completed, the neural network 
should be able to reproduce a specific route following a 
starting pattern. 

The two-layer structure of the SSM model is reminis- 
cent of the projection network implementation of the 
normal form projection algorithm (NFPA) 0. In that 
model, the dynamics of the network is cast in terms of 
the normal form equations which are written for ampli- 
tudes of certain normal forms which correspond to dif- 
ferent patterns stored in the system. The normal form 
dynamics can be chosen to follow certain dynamical rules, 
for example, in Q it was shown that a Hopfield-type net- 
work with improved capacity can be built using this ap- 
proach. Furthermore, in |1 it was proposed that specific 
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choices of the coupling matrix for the normal form dy- 
namics can lead to multi-stability among more complex 
attracting sets than simple fixed points, such as limit cy- 
cles or even chaotic attractors. As we will see below, the 
model of SSM after learning is completed can be viewed 
as a variant of the NFPA with m specific choice of normal 
form dynamics corresponding to the winncrlcss competi- 
tion among different patterns. 

Consider a two-level network of N s sensory neurons 
(SN) Xi and N p principal neurons eij. Similar to the pro- 
jection network model ||, we assume that sensory neu- 
rons do not have their own dynamics and are slaved to 
either external stimuli in the learning (or storing) regime, 
or to the PNs in the retrieval regime. In the learning 
regime, Xi = 7, where {Jj} is a binary input pattern 
consisting of O's and l's. During the retrieval phase, 
Xi = J2^Zi Pij a j: where Pjj is the N s x N p projection 
matrix of connections between SNs and PNs. 

The PNs are driven by SNs during the learning phase, 
but they also have their own dynamics controlled by in- 
hibitory inter-connections (see above) . After the learning 
is finished, the direct driving from SNs is disconnected. 
The equations for the amplitudes of PNs, a.;, read 

d. t — a t - a,i Vijdj + otdi P^Xj + £(t), (1) 

3=1 3=1 

where a ^ in the learning phase, and a — in the re- 
trieval phase. We use the transposed projection matrix 
assuming that the coupling between SNs and PNs is 
bi-directional and symmetric. The last term in the r.h.s. 
of Eq. (Q) represents small positive external perturbations 
which we model as white noise uniformly distributed be- 
tween and cr, however in reality it can represent input 
signals from other parts of the brain which control learn- 
ing and retrieval dynamics. 

After a certain pattern is presented to the model, the 
sensory stimuli reset the state of the PN layer accord- 
ing to the projection rule <Zj = Ylj^i Pij x ii but then <ij 
change according to Eq.(|l]). 

In addition to the dynamics of SNs and PNs during 
learning and retrieval phases, we need to introduce two 
learning processes: (i) forming the projection matrix Py 
which is responsible for connecting a group of sensory 
neurons of the first layer corresponding to a certain stored 
pattern to a single principal neuron which represents this 
pattern at the PN level; (ii) learning of the competition 
matrix Vij which is responsible for the temporal/logical 
ordering of the sequential memory. 

Projection matrix. The slow learning dynamics of the 
projection matrix is controlled by the following equation 

Pij = eatipxj - Pij). (2) 

with e< 1. We assume that initially all Py connections 
are nearly identical Pij = 1 + ?/y , where rjij are small 



random perturbations, J2jVij — 0, (vfj) = Vo 1- Ad- 
ditionally, we assume that initially matrix Vij is purely 
competitive: Vu = 1 and Vij = Vq > 1 for i ^ j. 

Consider a scenario when we want to "memorize" a 
certain pattern A in our projection matrix. We apply a 
set of inputs Ai corresponding to the pattern A to the 
SNs. As before, we assume that external stimuli ren- 
der the SNs in one of two states: excited (Ai = 1) and 
quiescent (Ai = 0) . The initial state of the PN layer is 
fully excited (ai(0) = PijAj). According to the com- 
petitive nature of interaction of PNs after a short tran- 
sient, only one of them (neuron A) which corresponds 
to maximum a^(0) remains excited and others become 
quiescent (inhibited). Which neuron becomes "responsi- 
ble" for the pattern A is actually random, as it depends 
on the initial projection matrix Py. As it follows from 
Eq.(||), for small e 'synapses' of suppressed PNs don't 
change, whereas synapses of the (single) excited neuron 
evolve such that the connections between excited SNs 
and PNs neurons amplify towards [3 > 1, and connection 
between excited PNs and quiescent SNs decay to zero 
(see Figure |[a). As a result, the first input pattern will 
be "recorded" in one of the rows of the matrix Py , while 
other rows will remain almost unchanged. 

Now suppose that we want to record a second pattern 
different from the first one. We can repeat the procedure 
described in the previous paragraph, namely, apply ex- 
ternal stimuli (pattern B) to the SNs, "project" them to 
the initial state of the PN layer (a.i(0) = Ylj PijPj)> and 
let the system evolve. Since synaptic connections from 
SNs suppressed by the first pattern to neuron A have 
been eliminated, a new set of stimuli corresponding to 
pattern B will excite neuron A weaker than most of the 
others, and competition will lead to selection of one prin- 
cipal neuron B different from neuron A. In such a way 
we can record as many patterns as there are PNs. 

Competition matrix. The sequential order of patterns 
recorded in the projection network is determined by the 
competition matrix Vij . Initially it is set to Vij = Vq > 1 
for i =/= j and Vu = 1 which corresponds to winner-take- 
all competition. The goal of sequential spatial learning 
is to record the transition of pattern A to pattern B in 
the form of suppressing the competition matrix element 
Vba- The slow dynamics of the non-diagonal elements 
of the competition matrix are controlled by the delay- 
differential equation 

Vij = ea t (t)a 3 (t - t)(V x - Vij). (3) 

As seen from Eq.(|J), only the matrix elements corre- 
sponding to ai(t) ^ and dj(t — r) ^ 0, are changing 
towards the asymptotic value V\ < 1 corresponding to 
the desired transition. Since most of the time (except 
for short transients) only one of the principal neurons 
is excited, only one of the connections Vij is changing 
at any time (see Figure |l|,b). As a result, an arbitrary 
(non-repeating) sequence of patterns can be recorded. If, 
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after a series of non-repeating patterns, we show the first 
pattern again, the "loop" of heteroclinic connections will 
be closed and the system will be able to reproduce a re- 
peating sequence of patterns in a cyclic manner. 

If the dimension of the secondary layer N s permits, it 
is easy to record into the network more than one sequence 
of patterns. To avoid a spurious connection between the 
sequences, the time interval between the last pattern of 
the first sequence and the first pattern of the second se- 
quence should be greater than r. 
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FIG. 1: The strengths of the connection coefficients between 
the sensory and the principal layers (a) and within the prin- 
cipal layer (b). Parameters of simulations: N s = 588, JV P = 
10, a = 1, (3 = 2.5, Vi = 0.9, e = 0.01, a = 10~ 4 , r = 480 

In Figure |l] we show the simulation results for a slow 
dynamics of weights and Vy during a learning phase 
in a network with 588 sensory and 10 principal neurons 
for e = 0.01. As stored patterns Zj we take ten digits 0,...9 
represented as 21x28 pixel dithered images. Two loop se- 
quences of patterns have been presented: "0" , "1" , "2" , 
and "6" , "7" , "8" , "9" . Note that these images are not 
precisely orthogonal to each other, and yet the system is 
able to associate them to different PNs. While a certain 
pattern is presented to the SN layer, certain matrix coeffi- 
cients Pij decay, some other (connecting excited neurons 
of the sensory layer and a single excited neuron of the PN 
layer) approach 2.5 and remaining connections remain al- 
most unchanged. After a switch from one pattern to the 
next in a sequence the corresponding matrix coefficient 
Vij decays to a low value V\ =0.9. 

Figure § shows the state of the projection matrix after 
all seven patterns have been recorded into the memory. 
The high-contrast "barcode" rows correspond to memo- 
rized patterns and dim gray rows correspond to yet avail- 
able principal neurons. 

Now, presenting a test pattern T "resembling" one of 




FIG. 2: The gray-level coded structure of the projection ma- 
trix Pij after seven distinct patterns have been recorded. 
Black-and-white rows correspond to recorded patterns (shown 
on the right) and gray rows corresponds to available PNs. Pa- 
rameters of simulations are the same as in Fig.nl 



the recorded patterns to the sensory layer (aci(0) = T(i), 
aj(0) = Pij T Tj), will initiate a periodic sequence of 
patterns corresponding to the previously recording se- 
quence recorded in the network. Figure |^,a shows the 
behavior of the principal neurons after two different ini- 
tial patterns have been presented, one resembling digit 
"0" and another resembling digit "6" . In both cases, the 
system quickly settled onto a cyclic generation of patterns 
associated with a given test pattern. At any given time 
except for a short transient time between the patterns, 
only a single principal neuron is "on" , which corresponds 
to a particular pattern. The order in which the princi- 
pal neurons are turned on is completely determined by 
the structure of the WLC matrix Vij. The duration of 
each "state" is determined by the magnitude of external 
perturbations a. For a = 0, the system would asymp- 
totically approach the separatrices and so the durations 
of each state would grow indefinitely. For a finite <r, the 
duration of each state scales as — log a. In the above ex- 
ample, patterns are retrieved from the spatial memory 
periodically in time. However, it may be desirable for a 
system to be able to control the duration of individual 
patterns in the sequence. This can be easily achieved by 
modulating the overall strength of inhibitory connections 
Vij+A(t). While A(i) > 1-Vi, all fixed points are stable 
nodes, and so a single principle neuron keeps firing. In 
order to advance to the next patter, A(t) is suppressed to 
zero for a short period of time (0(— logo - )). An example 
of such non-periodic retrieval of images is shown in Fig. 
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FIG. 3: Amplitudes of principal neurons during the mem- 
ory retrieval phase, a - periodic retrieval, two different test 
patterns presented, b - aperiodic retrieval with modulated in- 
hibition. Parameters of simulations are the same as in Figjl]. 



|b. 

In conclusion, we demonstrated the new principle of 
operation for the sequential spatial memory which is 
based on the winnerless competition. This principle is 
embodied in the two-layer neuronal structure with the 
first layer serving as a sensory input for the second layer 
which performs winnerless competition among represen- 
tative principal neurons. We introduced the learning 
rules for the projection and the competition matrices 
which lead naturally to the desired function of the net- 
work. We also demonstrated that external perturbations 
can influence the timing of the transitions among the 
stored patterns, however, the sequence of patterns is 
robust against external perturbations. The model can 



operate in two regimes: externally-timed switching con- 
trolled by global modulation of inhibitory connections 
and spontaneous periodic switching between patterns. 
The latter can be relevant for route replays during sleep. 
Of course, our model only describes a generic mechanism 
of sequential spatial memory, in real biological systems 
neurons generate non-stationary spike trains and synap- 
tic dynamics is timing-dependent (see j7],[l(J). Moreover, 
instead of a single PN a given pattern can be represented 
by a group of neurons which would increase the struc- 
tural stability of the memory. All these generalizations 
will be addressed in our future work. 
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